On the Importance of Calibration in Semi-supervised Learning

State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling. During pseudo-labeling, the model's predictions on unlabeled data are used for training and thus, model calibration is important in mitigating confirmation bias. Yet, many SOTA methods are optimized for model performance, with little focus directed to improve model calibration. In this work, we empirically demonstrate that model calibration is strongly correlated with model performance and propose to improve calibration via approximate Bayesian techniques. We introduce a family of new SSL models that optimizes for calibration and demonstrate their effectiveness across standard vision benchmarks of CIFAR-10, CIFAR-100 and ImageNet, giving up to 15.9% improvement in test accuracy. Furthermore, we also demonstrate their effectiveness in additional realistic and challenging problems, such as class-imbalanced datasets and in photonics science.Comment: 24 page

Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries

Deep ensembles (DE) have been successful in improving model performance by learning diverse members via the stochasticity of random initialization. While recent works have attempted to promote further diversity in DE via hyperparameters or regularizing loss functions, these methods primarily still rely on a stochastic approach to explore the hypothesis space. In this work, we present Multi-Symmetry Ensembles (MSE), a framework for constructing diverse ensembles by capturing the multiplicity of hypotheses along symmetry axes, which explore the hypothesis space beyond stochastic perturbations of model weights and hyperparameters. We leverage recent advances in contrastive representation learning to create models that separately capture opposing hypotheses of invariant and equivariant functional classes and present a simple ensembling approach to efficiently combine appropriate hypotheses for a given task. We show that MSE effectively captures the multiplicity of conflicting hypotheses that is often required in large, diverse datasets like ImageNet. As a result of their inherent diversity, MSE improves classification performance, uncertainty quantification, and generalization across a series of transfer tasks.Comment: Camera Ready Revision. ICML 202

Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies

Transformations based on domain expertise (expert transformations), such as random-resized-crop and color-jitter, have proven critical to the success of contrastive learning techniques such as SimCLR. Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned. However for imagery data, so far none of these view-generation methods has been able to outperform expert transformations. In this work, we tackle a different question: instead of replacing expert transformations with generated views, can we constructively assimilate generated views with expert transformations? We answer this question in the affirmative and propose a view generation method and a simple, effective assimilation method that together improve the state-of-the-art by up to ~3.6% on three different datasets. Importantly, we conduct a detailed empirical study that systematically analyzes a range of view generation and assimilation methods and provides a holistic picture of the efficacy of learned views in contrastive representation learning.Comment: Accepted at Generative Models for Computer Vision Workshop 202

SyncGC: A Synchronized Garbage Collection Technique for Reducing Tail Latency in Cassandra

Data-center applications running on distributed databases often suffer from unexpectedly high response time fluctuation which is caused by long tail latency. In this paper, we find that long tail latency of user writes is mainly created by the interference with garbage collection (GC) tasks running in various system layers. In order to address the tail latency problem, we propose a synchronized garbage collection technique, called SyncGC. By scheduling multiple GC instances to execute in sync with each other in an overlapped manner, SyncGC prevents user requests from being interfered with GC instances, thereby minimizing their negative impacts on tail latency. Our experimental results with Cassandra showthat SyncGC reduces the 99.99th-percentile tail latency and the maximum latency by 35% and 37%, on average, respectively.N

Optimization of Lead Insulations for HVDC Converter Transformers

This paper explains design considerations for lead and lead-exit insulations of HVDC converter transformer for the purpose of automated design optimization. A HVDC converter transformer should at least deal with AC, DC, and DC polarity reversal stresses, which complicate designing process of insulation structures. Design considerations for those insulations are the types of electrical stresses and design parameters of insulation shapes. The design objective of the optimization is to ensure insulation robustness with reduced size of insulation structures. Optimization results showed 16.72% and 29.07% of length reduction with sufficient robustness, for lead and lead-exit insulations, respectively

Phase evolution of Cu2ZnSnS4 (CZTS) kesterite thin films during the sulfurization process

The development of high efficiency Cu2ZnSnS4 (CZTS) thin film solar cells depends on the synthesis of phase pure CZTS films. Due to the narrow phase window, it is difficult to prepare CZTS films with a pure kesterite phase (i.e., a film that lacks traces of detrimental secondary phases). For this, phase evolution of kesterite CZTS in films must be systematically studied. In the present study detailed phase evolution of kesterite CZTS via the sulfurization of stacked Cu/SnS2/ZnS precursor was examined. The phase evolution of kesterite CZTS was analyzed during the temperature-dependent sulfurization of stacked precursors. In addition to traditional X-ray diffraction and Raman analysis, transmission electron microscopy (TEM) characterization was used to study the phase evolution and trace locations of secondary phases in the sulfurized films. Furthermore, based on the experimental evidences the reaction pathways for the formation of CZTS kesterite phase and plausible phase evolution mechanism are proposed. (C) 2015 Published by Elsevier Ltd and Techna Group S.r.l11591sciescopu